{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "unsplash-image-search.ipynb",
      "provenance": [],
      "collapsed_sections": [
        "Ons94QRxCQR2",
        "lJVrkmy6DVj2",
        "ujCKerTnFBk4",
        "lsdZUxzJK_af"
      ]
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "97qkK30wAzXb"
      },
      "source": [
        "# Unsplash Image Search\n",
        "\n",
        "Using this notebook you can search for images from the [Unsplash Dataset](https://unsplash.com/data) using natural language queries. The search is powered by OpenAI's [CLIP](https://github.com/openai/CLIP) neural network.\n",
        "\n",
        "This notebook uses the precomputed feature vectors for almost 2 million images from the full version of the [Unsplash Dataset](https://unsplash.com/data). If you want to compute the features yourself, see [here](https://github.com/haltakov/natural-language-image-search#on-your-machine).\n",
        "\n",
        "This project was created by [Vladimir Haltakov](https://twitter.com/haltakov) and the full code is open-sourced on [GitHub](https://github.com/haltakov/natural-language-image-search)."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Ons94QRxCQR2"
      },
      "source": [
        "## Setup Environment\n",
        "\n",
        "In this section we will setup the environment."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DYdIafWsOOUV"
      },
      "source": [
        "First we need to install CLIP and then make sure that we have torch 1.7.1 with CUDA support."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "djgE7IjbV3sv"
      },
      "source": [
        "!pip install git+https://github.com/openai/CLIP.git\n",
        "!pip install torch==1.7.1+cu101 torchvision==0.8.2+cu101 -f https://download.pytorch.org/whl/torch_stable.html"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "B7_Sk-T7DBEm"
      },
      "source": [
        "We can now load the pretrained public CLIP model."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "_6FzbzS6W1R5"
      },
      "source": [
        "import clip\n",
        "import torch\n",
        "\n",
        "# Load the open CLIP model\n",
        "device = \"cuda\" if torch.cuda.is_available() else \"cpu\"\n",
        "model, preprocess = clip.load(\"ViT-B/32\", device=device)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lJVrkmy6DVj2"
      },
      "source": [
        "## Download the Precomputed Data\n",
        "\n",
        "In this section the precomputed feature vectors for all photos are downloaded."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "18alAEjEOdSC"
      },
      "source": [
        "In order to compare the photos from the Unsplash dataset to a text query, we need to compute the feature vector of each photo using CLIP. This is a time consuming task, so you can use the feature vectors that I precomputed and uploaded to Google Drive (with the permission from Unsplash). If you want to compute the features yourself, see [here](https://github.com/haltakov/natural-language-image-search#on-your-machine).\n",
        "\n",
        "We need to download two files:\n",
        "* `photo_ids.csv` - a list of the photo IDs for all images in the dataset. The photo ID can be used to get the actual photo from Unsplash.\n",
        "* `features.npy` - a matrix containing the precomputed 512 element feature vector for each photo in the dataset.\n",
        "\n",
        "The files are available on [Google Drive](https://drive.google.com/drive/folders/1WQmedVCDIQKA2R33dkS1f980YsJXRZ-q?usp=sharing)."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "BAb15OJQZRkt"
      },
      "source": [
        "from pathlib import Path\n",
        "\n",
        "# Create a folder for the precomputed features\n",
        "!mkdir unsplash-dataset\n",
        "\n",
        "# Download from Github Releases\n",
        "if not Path('unsplash-dataset/photo_ids.csv').exists():\n",
        "  !wget https://github.com/haltakov/natural-language-image-search/releases/download/1.0.0/photo_ids.csv -O unsplash-dataset/photo_ids.csv\n",
        "\n",
        "if not Path('unsplash-dataset/features.npy').exists():\n",
        "  !wget https://github.com/haltakov/natural-language-image-search/releases/download/1.0.0/features.npy -O unsplash-dataset/features.npy\n",
        "  "
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "TVjuUh6oEtPt"
      },
      "source": [
        "After the files are downloaded we need to load them using `pandas` and `numpy`."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "GQHcmMo1Ztjz",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "8b77c4b9-a194-4c11-fb70-828e6a6cae6b"
      },
      "source": [
        "import pandas as pd\n",
        "import numpy as np\n",
        "\n",
        "# Load the photo IDs\n",
        "photo_ids = pd.read_csv(\"unsplash-dataset/photo_ids.csv\")\n",
        "photo_ids = list(photo_ids['photo_id'])\n",
        "\n",
        "# Load the features vectors\n",
        "photo_features = np.load(\"unsplash-dataset/features.npy\")\n",
        "\n",
        "# Convert features to Tensors: Float32 on CPU and Float16 on GPU\n",
        "if device == \"cpu\":\n",
        "  photo_features = torch.from_numpy(photo_features).float().to(device)\n",
        "else:\n",
        "  photo_features = torch.from_numpy(photo_features).to(device)\n",
        "\n",
        "# Print some statistics\n",
        "print(f\"Photos loaded: {len(photo_ids)}\")"
      ],
      "execution_count": 7,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Photos loaded: 1981161\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ujCKerTnFBk4"
      },
      "source": [
        "## Define Functions\n",
        "\n",
        "Some important functions for processing the data are defined here.\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pYVNtF-JFtfj"
      },
      "source": [
        "The `encode_search_query` function takes a text description and encodes it into a feature vector using the CLIP model."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "d0hmOh3qbcxK"
      },
      "source": [
        "def encode_search_query(search_query):\n",
        "  with torch.no_grad():\n",
        "    # Encode and normalize the search query using CLIP\n",
        "    text_encoded = model.encode_text(clip.tokenize(search_query).to(device))\n",
        "    text_encoded /= text_encoded.norm(dim=-1, keepdim=True)\n",
        "\n",
        "  # Retrieve the feature vector\n",
        "  return text_encoded"
      ],
      "execution_count": 8,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Vh1yyJtEGCAX"
      },
      "source": [
        "The `find_best_matches` function compares the text feature vector to the feature vectors of all images and finds the best matches. The function returns the IDs of the best matching photos."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "3TcmI5KIbe5F"
      },
      "source": [
        "def find_best_matches(text_features, photo_features, photo_ids, results_count=3):\n",
        "  # Compute the similarity between the search query and each photo using the Cosine similarity\n",
        "  similarities = (photo_features @ text_features.T).squeeze(1)\n",
        "\n",
        "  # Sort the photos by their similarity score\n",
        "  best_photo_idx = (-similarities).argsort()\n",
        "\n",
        "  # Return the photo IDs of the best matches\n",
        "  return [photo_ids[i] for i in best_photo_idx[:results_count]]"
      ],
      "execution_count": 9,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "gEmt0F4iHbL0"
      },
      "source": [
        "The `display_photo` function displays a photo from Unsplash given its ID and link to the original photo on Unsplash. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "RC4HD8cBYOon"
      },
      "source": [
        "from IPython.display import Image\n",
        "from IPython.core.display import HTML\n",
        "\n",
        "def display_photo(photo_id):\n",
        "  # Get the URL of the photo resized to have a width of 320px\n",
        "  photo_image_url = f\"https://unsplash.com/photos/{photo_id}/download?w=320\"\n",
        "\n",
        "  # Display the photo\n",
        "  display(Image(url=photo_image_url))\n",
        "\n",
        "  # Display the attribution text\n",
        "  display(HTML(f'Photo on <a target=\"_blank\" href=\"https://unsplash.com/photos/{photo_id}\">Unsplash</a> '))\n",
        "  print()"
      ],
      "execution_count": 10,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_3ojinZ0JYBC"
      },
      "source": [
        "Putting it all together in one function."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "LvUcljF5JcRn"
      },
      "source": [
        "def search_unslash(search_query, photo_features, photo_ids, results_count=3):\n",
        "  # Encode the search query\n",
        "  text_features = encode_search_query(search_query)\n",
        "\n",
        "  # Find the best matches\n",
        "  best_photo_ids = find_best_matches(text_features, photo_features, photo_ids, results_count)\n",
        "\n",
        "  # Display the best photos\n",
        "  for photo_id in best_photo_ids:\n",
        "    display_photo(photo_id)\n"
      ],
      "execution_count": 11,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xbym_cYJJH6v"
      },
      "source": [
        "## Search Unsplash\n",
        "\n",
        "Now we are ready to search the dataset using natural language. Check out the examples below and feel free to try out your own queries."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-RmOFAq5NtlI"
      },
      "source": [
        "### \"Two dogs playing in the snow\""
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "CF7HuxAlFNXT",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 779
        },
        "outputId": "fb778e0c-d85b-4146-9c7f-ef3cd7781e9a"
      },
      "source": [
        "search_query = \"Two dogs playing in the snow\"\n",
        "\n",
        "search_unslash(search_query, photo_features, photo_ids, 3)"
      ],
      "execution_count": 21,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/html": [
              "<img src=\"https://unsplash.com/photos/cBAcMcwQ1h0/download?w=320\"/>"
            ],
            "text/plain": [
              "<IPython.core.display.Image object>"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/html": [
              "Photo on <a target=\"_blank\" href=\"https://unsplash.com/photos/cBAcMcwQ1h0\">Unsplash</a> "
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "text": [
            "\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "display_data",
          "data": {
            "text/html": [
              "<img src=\"https://unsplash.com/photos/jKDFcXwk5Cw/download?w=320\"/>"
            ],
            "text/plain": [
              "<IPython.core.display.Image object>"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/html": [
              "Photo on <a target=\"_blank\" href=\"https://unsplash.com/photos/jKDFcXwk5Cw\">Unsplash</a> "
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "text": [
            "\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "display_data",
          "data": {
            "text/html": [
              "<img src=\"https://unsplash.com/photos/rKxSxPAtW_E/download?w=320\"/>"
            ],
            "text/plain": [
              "<IPython.core.display.Image object>"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/html": [
              "Photo on <a target=\"_blank\" href=\"https://unsplash.com/photos/rKxSxPAtW_E\">Unsplash</a> "
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "text": [
            "\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "PtaYocbjN0VQ"
      },
      "source": [
        "### \"The word love written on the wall\""
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "OswqrzaeMy1J",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 557
        },
        "outputId": "32abb6c3-5f98-4302-a6d2-5524efdb211f"
      },
      "source": [
        "search_query = \"The word love written on the wall\"\n",
        "\n",
        "search_unslash(search_query, photo_features, photo_ids, 3)"
      ],
      "execution_count": 13,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/html": [
              "<img src=\"https://unsplash.com/photos/3KO3bRLxsag/download?w=320\"/>"
            ],
            "text/plain": [
              "<IPython.core.display.Image object>"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/html": [
              "Photo on <a target=\"_blank\" href=\"https://unsplash.com/photos/3KO3bRLxsag\">Unsplash</a> "
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "text": [
            "\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "display_data",
          "data": {
            "text/html": [
              "<img src=\"https://unsplash.com/photos/TXMJKoy-Q3k/download?w=320\"/>"
            ],
            "text/plain": [
              "<IPython.core.display.Image object>"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/html": [
              "Photo on <a target=\"_blank\" href=\"https://unsplash.com/photos/TXMJKoy-Q3k\">Unsplash</a> "
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "text": [
            "\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "display_data",
          "data": {
            "text/html": [
              "<img src=\"https://unsplash.com/photos/HgvuuKoOseA/download?w=320\"/>"
            ],
            "text/plain": [
              "<IPython.core.display.Image object>"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/html": [
              "Photo on <a target=\"_blank\" href=\"https://unsplash.com/photos/HgvuuKoOseA\">Unsplash</a> "
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "text": [
            "\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "sUdySrczN4ZX"
      },
      "source": [
        "### \"The feeling when your program finally works\""
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "SRyCZMHQMzOP",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 842
        },
        "outputId": "ebab05a9-99f5-4759-a582-230403b50343"
      },
      "source": [
        "search_query = \"The feeling when your program finally works\"\n",
        "\n",
        "search_unslash(search_query, photo_features, photo_ids, 3)"
      ],
      "execution_count": 14,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/html": [
              "<img src=\"https://unsplash.com/photos/AndE50aaHn4/download?w=320\"/>"
            ],
            "text/plain": [
              "<IPython.core.display.Image object>"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/html": [
              "Photo on <a target=\"_blank\" href=\"https://unsplash.com/photos/AndE50aaHn4\">Unsplash</a> "
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "text": [
            "\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "display_data",
          "data": {
            "text/html": [
              "<img src=\"https://unsplash.com/photos/I6-sHSAt9vc/download?w=320\"/>"
            ],
            "text/plain": [
              "<IPython.core.display.Image object>"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/html": [
              "Photo on <a target=\"_blank\" href=\"https://unsplash.com/photos/I6-sHSAt9vc\">Unsplash</a> "
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "text": [
            "\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "display_data",
          "data": {
            "text/html": [
              "<img src=\"https://unsplash.com/photos/02oFZfhfX_I/download?w=320\"/>"
            ],
            "text/plain": [
              "<IPython.core.display.Image object>"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/html": [
              "Photo on <a target=\"_blank\" href=\"https://unsplash.com/photos/02oFZfhfX_I\">Unsplash</a> "
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "text": [
            "\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "aR4aDfQYN8J1"
      },
      "source": [
        "### \"The Syndey Opera House and the Harbour Bridge at night\""
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "wWkWfHhnMzZe",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 395
        },
        "outputId": "5d1958b6-2b7e-408f-fda8-83f2e2f2fdf6"
      },
      "source": [
        "search_query = \"The Syndey Opera House and the Harbour Bridge at night\"\n",
        "\n",
        "search_unslash(search_query, photo_features, photo_ids, 3)"
      ],
      "execution_count": 15,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/html": [
              "<img src=\"https://unsplash.com/photos/beYNO-kn4-Y/download?w=320\"/>"
            ],
            "text/plain": [
              "<IPython.core.display.Image object>"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/html": [
              "Photo on <a target=\"_blank\" href=\"https://unsplash.com/photos/beYNO-kn4-Y\">Unsplash</a> "
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "text": [
            "\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "display_data",
          "data": {
            "text/html": [
              "<img src=\"https://unsplash.com/photos/cY8I_ncKezc/download?w=320\"/>"
            ],
            "text/plain": [
              "<IPython.core.display.Image object>"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/html": [
              "Photo on <a target=\"_blank\" href=\"https://unsplash.com/photos/cY8I_ncKezc\">Unsplash</a> "
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "text": [
            "\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "display_data",
          "data": {
            "text/html": [
              "<img src=\"https://unsplash.com/photos/OAzRalQb0F4/download?w=320\"/>"
            ],
            "text/plain": [
              "<IPython.core.display.Image object>"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/html": [
              "Photo on <a target=\"_blank\" href=\"https://unsplash.com/photos/OAzRalQb0F4\">Unsplash</a> "
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "text": [
            "\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lsdZUxzJK_af"
      },
      "source": [
        "## Combine Text and Photo Seach Queries\n",
        "\n",
        "This is another experiment to combine a text query with another photo."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_Yxox5LGfqEj"
      },
      "source": [
        "The idea here is to do a text search for a photo and then modify the search query by adding another photo to the search query in order to transfer some of the photo features to the search.\n",
        "\n",
        "This works by adding the features of the photo to the features of the text query. The photo features are multiplied with a weight in order to reduce the influence so that the text query is the main source.\n",
        "\n",
        "The results are somewhat sensitive to the prompt..."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ZEBq8_TeUOFm"
      },
      "source": [
        "def search_by_text_and_photo(query_text, query_photo_id, photo_weight=0.5):\n",
        "  # Encode the search query\n",
        "  text_features = encode_search_query(query_text)\n",
        "\n",
        "  # Find the feature vector for the specified photo ID\n",
        "  query_photo_index = photo_ids.index(query_photo_id)\n",
        "  query_photo_features = photo_features[query_photo_index]\n",
        "\n",
        "  # Combine the test and photo queries and normalize again\n",
        "  search_features = text_features + query_photo_features * photo_weight\n",
        "  search_features /= search_features.norm(dim=-1, keepdim=True)\n",
        "\n",
        "  # Find the best match\n",
        "  best_photo_ids = find_best_matches(search_features, photo_features, photo_ids, 1)\n",
        "\n",
        "  # Display the results\n",
        "  print(\"Test search result\")\n",
        "  search_unslash(query_text, photo_features, photo_ids, 1)\n",
        "\n",
        "  print(\"Photo query\")\n",
        "  display(Image(url=f\"https://unsplash.com/photos/{query_photo_id}/download?w=320\"))\n",
        "\n",
        "  print(\"Result for text query + photo query\")\n",
        "  display_photo(best_photo_ids[0])"
      ],
      "execution_count": 16,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "h1foDoFYgRPm"
      },
      "source": [
        "## Results Combining Text and Photo Seach Queries\n",
        "\n",
        "Now some results for combining text and photo queries"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "g0uuA3aYguZe"
      },
      "source": [
        "### Sydney Opera House + night photo"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 634
        },
        "id": "ruf7Tmy9LVor",
        "outputId": "9edbcaf3-e7f7-40ed-bbf5-997ede3179b4"
      },
      "source": [
        "search_by_text_and_photo(\"Sydney Opera house\", \"HSsOC5nqurA\")"
      ],
      "execution_count": 17,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Test search result\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "display_data",
          "data": {
            "text/html": [
              "<img src=\"https://unsplash.com/photos/QP-l1uE19iI/download?w=320\"/>"
            ],
            "text/plain": [
              "<IPython.core.display.Image object>"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/html": [
              "Photo on <a target=\"_blank\" href=\"https://unsplash.com/photos/QP-l1uE19iI\">Unsplash</a> "
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "text": [
            "\n",
            "Photo query\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "display_data",
          "data": {
            "text/html": [
              "<img src=\"https://unsplash.com/photos/HSsOC5nqurA/download?w=320\"/>"
            ],
            "text/plain": [
              "<IPython.core.display.Image object>"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "text": [
            "Result for text query + photo query\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "display_data",
          "data": {
            "text/html": [
              "<img src=\"https://unsplash.com/photos/KCEUHBb4wjc/download?w=320\"/>"
            ],
            "text/plain": [
              "<IPython.core.display.Image object>"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/html": [
              "Photo on <a target=\"_blank\" href=\"https://unsplash.com/photos/KCEUHBb4wjc\">Unsplash</a> "
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "text": [
            "\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "z_A9ui7ugyM8"
      },
      "source": [
        "### Sydney Opera House + mist photo"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 662
        },
        "id": "NXba1meDdfAt",
        "outputId": "5748190d-0a2d-4908-fc97-de205fc0ff70"
      },
      "source": [
        "search_by_text_and_photo(\"Sydney Opera house\", \"MaerUPAjPbs\")"
      ],
      "execution_count": 18,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Test search result\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "display_data",
          "data": {
            "text/html": [
              "<img src=\"https://unsplash.com/photos/QP-l1uE19iI/download?w=320\"/>"
            ],
            "text/plain": [
              "<IPython.core.display.Image object>"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/html": [
              "Photo on <a target=\"_blank\" href=\"https://unsplash.com/photos/QP-l1uE19iI\">Unsplash</a> "
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "text": [
            "\n",
            "Photo query\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "display_data",
          "data": {
            "text/html": [
              "<img src=\"https://unsplash.com/photos/MaerUPAjPbs/download?w=320\"/>"
            ],
            "text/plain": [
              "<IPython.core.display.Image object>"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "text": [
            "Result for text query + photo query\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "display_data",
          "data": {
            "text/html": [
              "<img src=\"https://unsplash.com/photos/GSXMuGwY2l4/download?w=320\"/>"
            ],
            "text/plain": [
              "<IPython.core.display.Image object>"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/html": [
              "Photo on <a target=\"_blank\" href=\"https://unsplash.com/photos/GSXMuGwY2l4\">Unsplash</a> "
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "text": [
            "\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "SQcwhrAOg0pg"
      },
      "source": [
        "### Sydney Opera House + rain photo"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        },
        "id": "rjq8GcBOfNDA",
        "outputId": "4da7b557-8953-4b60-84cd-3551b2bf03d7"
      },
      "source": [
        "search_by_text_and_photo(\"Sydney Opera house\", \"1pNBJ2zUfn4\", 0.4)"
      ],
      "execution_count": 19,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Test search result\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "display_data",
          "data": {
            "text/html": [
              "<img src=\"https://unsplash.com/photos/QP-l1uE19iI/download?w=320\"/>"
            ],
            "text/plain": [
              "<IPython.core.display.Image object>"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/html": [
              "Photo on <a target=\"_blank\" href=\"https://unsplash.com/photos/QP-l1uE19iI\">Unsplash</a> "
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "text": [
            "\n",
            "Photo query\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "display_data",
          "data": {
            "text/html": [
              "<img src=\"https://unsplash.com/photos/1pNBJ2zUfn4/download?w=320\"/>"
            ],
            "text/plain": [
              "<IPython.core.display.Image object>"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "text": [
            "Result for text query + photo query\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "display_data",
          "data": {
            "text/html": [
              "<img src=\"https://unsplash.com/photos/6WuX1eYOciE/download?w=320\"/>"
            ],
            "text/plain": [
              "<IPython.core.display.Image object>"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/html": [
              "Photo on <a target=\"_blank\" href=\"https://unsplash.com/photos/6WuX1eYOciE\">Unsplash</a> "
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "text": [
            "\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dXXsOKhUg3nU"
      },
      "source": [
        "### Sydney Opera House + sea photo"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 607
        },
        "id": "kQiQ4w-HfeL6",
        "outputId": "6e50498d-9383-470c-c7e0-c20d67a50892"
      },
      "source": [
        "search_by_text_and_photo(\"Sydney Opera house\", \"jnBDclcdZ7A\", 0.4)"
      ],
      "execution_count": 20,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Test search result\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "display_data",
          "data": {
            "text/html": [
              "<img src=\"https://unsplash.com/photos/QP-l1uE19iI/download?w=320\"/>"
            ],
            "text/plain": [
              "<IPython.core.display.Image object>"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/html": [
              "Photo on <a target=\"_blank\" href=\"https://unsplash.com/photos/QP-l1uE19iI\">Unsplash</a> "
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "text": [
            "\n",
            "Photo query\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "display_data",
          "data": {
            "text/html": [
              "<img src=\"https://unsplash.com/photos/jnBDclcdZ7A/download?w=320\"/>"
            ],
            "text/plain": [
              "<IPython.core.display.Image object>"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "text": [
            "Result for text query + photo query\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "display_data",
          "data": {
            "text/html": [
              "<img src=\"https://unsplash.com/photos/2gwhorA9tic/download?w=320\"/>"
            ],
            "text/plain": [
              "<IPython.core.display.Image object>"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/html": [
              "Photo on <a target=\"_blank\" href=\"https://unsplash.com/photos/2gwhorA9tic\">Unsplash</a> "
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "text": [
            "\n"
          ],
          "name": "stdout"
        }
      ]
    }
  ]
}